Increasing Skew Insensitivity of Decision Trees with Hellinger Distance

Learning from unbalanced datasets presents a convoluted problem in which traditional learning algorithms typically perform poorly. The heuristics used in learning tend to favor the larger, less important classes in such problems. While other methods, like sampling, have been introduced to combat imbalance, these tend to be computationally expensive. This paper proposes Hellinger distance as a means to induce decision trees. We demonstrate that Hellinger distance is skew insensitive, especially as compared to information gain. This leads the Hellinger Distance Decision Tree (HDDT) induction algorithm and we demonstrate significant performance improvement on problems within the realm of class imbalance.